What score markings can say of the synergy between expressive timing and loudness

Background Performance gestures are often realized along different expressive dimensions (e.g., a change in the dynamics might be emphasized by a change of tempo) and they may also be affected by constraints imposed by a score. Elucidating how such interactions and score dependencies can be better modeled is a key element to further understanding the characterization of both individual and shared performance approaches. Aims We examine possible interactions between tempo, loudness and specific score markings over a set of performances. We hypothesize that tempo and loudness can be better predicted at score markings when including contextual information from two bars before the score markings and when combining them as complementary expressive features, instead of when considering them as isolated. In particular, we examine how tempo may contribute to the prediction of loudness at dynamic score markings (e.g., pp, f), and how loudness may contribute to the prediction of tempo at tempo score markings (e.g., lento, moderato). Method We conduct two experiments. In our first experiment (E1) we model collective approaches to the use of tempo and loudness. In our second experiment (E2) we model individual approaches. In both experiments our goal is predicting tempo or loudness for a specific piece based on how a group of performers (E1), or each individual performer (E2), played the rest of pieces in the corpus. We use a dataset of recordings of 26 different Chopin Mazurkas played by 11 pianists containing tempo, loudness and score markings annotations (Kosta et al., 2016). We collect a total of 317 dynamic markings and 109 tempo markings. Following Kosta et al. (2016), the score features proposed are: Marking at which either loudness or tempo is predicted, Previous marking, Next Marking, Possible additional marking, Distance in beats to previous marking, Distance in beats to next marking. In addition to the score based features, we propose the following performance features corresponding to two bars previous to each marking: Normalized Inter Beat Intervals (IBI), measured in seconds, Normalized Inter Beat Loudness (IBL), measured in sones. We train our models to predict the mean of the IBIs and IBLs of the bar at which tempo or dynamic markings are annotated. We experiment with Multi-Layer Perceptron, Random Forests and K-Nearest Neighbors and tune their hyper-parameters with exhaustive grid-search after applying the jack-knifing technique. We consider the following versions of the feature set: #S (only score based features), #L (#S + previous two bars IBL), #T (#S + previous two bars IBI), #A (#S + previous two bars IBI + IBL). We evaluate these models by measuring the mean squared error between the predicted values and the true values. Finally, we choose the best performing models and calculate significance using the Wilcoxon test. Results E1 shows significant improvements at tempo markings (p=0.015) when adding tempo features (#T) to score features (#S) but no improvements between #A and #T features models (p=0.771). In the case of dynamic markings, we observe improvements (p=0.017) when adding loudness features (#L) to score features (#S) and marginal improvements when adding tempo (#A) to loudness (#L) features (p=0.049). E2, at tempo markings, shows no improvements between #T and #S predictions and, in most models, no improvements when combining tempo and loudness features (#A). At dynamic markings, we observe that #L improves the predictions of #S (p=0.004) but no improvements when combining loudness and tempo features (#A). Our results also show that the predictions obtained on E1 are better than the predictions on E2. ConclusionsOur results indicate that loudness is, in most cases, better predicted when including performance features preceding dynamicmarkings and that individual tempo predictions at tempo markings are sensitive to IBIs variance across performances. We found noevidence for an interaction between tempo and loudness at dynamic or tempo markings. These appear not to be dependent on sharedor individual stylistic approaches. These results could be confirmed by studying alternative features and methods as well as byexamining larger datasets. Future work will address such potential interactions by studying them across entire performances usingsequential data models. ReferencesKosta, K., Ramírez, R., Bandtlow, O. F., & Chew, E. (2016). Mapping between dynamic markings and performed loudness: amachine learning approach. Journal of Mathematics and Music, 10(2), 149–172.